Electronic device and method for securing personal information included in image

ABSTRACT

An electronic device according to an embodiment includes: a camera including an image sensor; a communication module; a memory; and a processor operationally connected to the camera, the communication module, and the memory. The memory may store instructions that, when executed, instruct the processor to: acquire a first image including an external object by using the camera; acquire a second image including less information than the first image by using at least a part of the first image; transmit the second image to an external electronic device by using the communication module; receive security information corresponding to the second image from the external electronic device by using the communication module; and secure the first image, at least partially based on the security information. In addition, various other embodiments are possible.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. 119 toKorean Patent Application No. 10-2019-0035716, filed on Mar. 28, 2019,in the Korean Intellectual Property Office, the disclosure of which isherein incorporated by reference in its entirety.

BACKGROUND 1) Field

One or more embodiments generally relate to an electronic device and amethod for securing personal information included in an image.

2) Description of Related Art

Electronic devices, such as smart phones, tablet PCs, portablemultimedia players (PMPs), personal digital assistants (PDAs), laptoppersonal computers (laptop PCs), wrist watches, or head-mounteddisplays, may include one or more cameras, and may capture images byusing the cameras.

The user can upload images captured by the camera of an electronicdevice to an external electronic device, such as an electronic device orserver of a social network service (SNS).

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

With the increasing popularity of high-speed high-capacity wirelesscommunication deployed on the above mentioned electronic devices, highquality original images (e.g. raw images) captured by the aforementionedcameras may be uploaded to an SNS server.

The server or the external electronic device may apply an image qualityenhancement algorithm to the high quality original image received fromthe electronic device to enhance the quality of the image, and mayprovide the image having the enhanced quality back to the electronicdevice.

If the user of the electronic device uploads the image to the externalelectronic device or shares the image with another user through thesocial network service, personal information may be exposed or leaked toother people regardless of the user's intension. For example, one of thehigh-quality images may include the user's personal information (e.g.fingerprint information, iris information, or face information).

Recently, illegal hacking of servers has frequently occurred, and theinformation illegally obtained through hacking may cause great damage tothe user.

According to an embodiment, an electronic device may include: a cameraincluding an image sensor; a communication module; a memory; and aprocessor operationally connected to the camera, the communicationmodule, and the memory. The memory may store instructions that, whenexecuted, cause the processor to: acquire a first image including anexternal object by using the camera; acquire a second image includingless information than the first image by using at least a part of thefirst image; transmit the second image to an external electronic deviceby using the communication module; receive security informationcorresponding to the second image from the external electronic device byusing the communication module; and secure the first image at leastpartially based on the security information.

According to an embodiment, a method for securing personal informationincluded in an image may include: acquiring a first image including anexternal object by using a camera that includes an image sensor;acquiring a second image including less information than the first imageby using at least a part of the first image; transmitting the secondimage to an external electronic device by using a communication module;receiving security information corresponding to the second image fromthe external electronic device by using the communication module; andsecuring the first image, at least partially based on the securityinformation.

According to an embodiment, an electronic device may include: acommunication module; a memory; and a processor operationally connectedto the communication module and the memory. The memory storesinstructions that, when executed, cause the processor to: receive animage from an external electronic device by using the communicationmodule; recognize at least one image segment in the image; determinesecurity information included in the image and associated with the atleast one image segment, at least partially based on a result of therecognition; and transmit the security information to the externalelectronic device by using the communication module.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the disclosure and its advantages,reference is now made to the following description taken in conjunctionwith the accompanying drawings, in which like reference numeralsrepresent like parts:

FIG. 1 is a block diagram of an electronic device in a networkenvironment according to an embodiment;

FIG. 2 is a block diagram illustrating a camera module according to anembodiment;

FIG. 3 is a block diagram illustrating operations of an electronicdevice and an external electronic device according to an embodiment;

FIG. 4 is a block diagram of electronic devices according to anembodiment;

FIG. 5 is a flowchart illustrating a method for securing personalinformation included in an image in an electronic device according to anembodiment;

FIG. 6 is a flowchart illustrating a method for securing a first imagein an electronic device according to an embodiment;

FIG. 7 is a flowchart illustrating a method for securing a first imagein an electronic device according to an embodiment;

FIG. 8 is a flowchart illustrating a method for securing personalinformation included in an image in a system according to an embodiment;and

FIG. 9A illustrates an example of a method for securing personalinformation according to an embodiment, FIG. 9B illustrates an exampleof a method for securing personal information according to anembodiment, and FIG. 9C illustrates an example of a method for securingpersonal information according to an embodiment.

DETAILED DESCRIPTION

Hereinafter, embodiments of the disclosure will be described in detailwith reference to accompanying drawings.

FIG. 1 is a block diagram illustrating an electronic device 101 in anetwork environment 100 according to various embodiments. Referring toFIG. 1, the electronic device 101 in the network environment 100 maycommunicate with an electronic device 102 via a first network 198 (e.g.,a short-range wireless communication network), or an electronic device104 or a server 108 via a second network 199 (e.g., a long-rangewireless communication network). According to an embodiment, theelectronic device 101 may communicate with the electronic device 104 viathe server 108. According to an embodiment, the electronic device 101may include a processor 120, memory 130, an input device 150, a soundoutput device 155, a display device 160, an audio module 170, a sensormodule 176, an interface 177, a haptic module 179, a camera module 180,a power management module 188, a battery 189, a communication module190, a subscriber identification module (SIM) 196, or an antenna module197. In some embodiments, at least one (e.g., the display device 160 orthe camera module 180) of the components may be omitted from theelectronic device 101, or one or more other components may be added inthe electronic device 101. In some embodiments, some of the componentsmay be implemented as single integrated circuitry. For example, thesensor module 176 (e.g., a fingerprint sensor, an iris sensor, or anilluminance sensor) may be implemented as embedded in the display device160 (e.g., a display).

The processor 120 may execute, for example, software (e.g., a program140) to control at least one other component (e.g., a hardware orsoftware component) of the electronic device 101 coupled with theprocessor 120, and may perform various data processing or computation.According to one embodiment, as at least part of the data processing orcomputation, the processor 120 may load a command or data received fromanother component (e.g., the sensor module 176 or the communicationmodule 190) in volatile memory 132, process the command or the datastored in the volatile memory 132, and store resulting data innon-volatile memory 134. According to an embodiment, the processor 120may include a main processor 121 (e.g., a central processing unit (CPU)or an application processor (AP)), and an auxiliary processor 123 (e.g.,a graphics processing unit (GPU), an image signal processor (ISP), asensor hub processor, or a communication processor (CP)) that isoperable independently from, or in conjunction with, the main processor121. Additionally or alternatively, the auxiliary processor 123 may beadapted to consume less power than the main processor 121, or to bespecific to a specified function. The auxiliary processor 123 may beimplemented as separate from, or as part of the main processor 121.

The auxiliary processor 123 may control at least some of functions orstates related to at least one component (e.g., the display device 160,the sensor module 176, or the communication module 190) among thecomponents of the electronic device 101, instead of the main processor121 while the main processor 121 is in an inactive (e.g., sleep) state,or together with the main processor 121 while the main processor 121 isin an active state (e.g., executing an application). According to anembodiment, the auxiliary processor 123 (e.g., an image signal processoror a communication processor) may be implemented as part of anothercomponent (e.g., the camera module 180 or the communication module 190)functionally related to the auxiliary processor 123.

The memory 130 may store various data used by at least one component(e.g., the processor 120 or the sensor module 176) of the electronicdevice 101. The various data may include, for example, software (e.g.,the program 140) and input data or output data for a command relatedthereto. The memory 130 may include the volatile memory 132 or thenon-volatile memory 134.

The program 140 may be stored in the memory 130 as software, and mayinclude, for example, an operating system (OS) 142, middleware 144, oran application 146.

The input device 150 may receive a command or data to be used by othercomponent (e.g., the processor 120) of the electronic device 101, fromthe outside (e.g., a user) of the electronic device 101. The inputdevice 150 may include, for example, a microphone, a mouse, a keyboard,or a digital pen (e.g., a stylus pen).

The sound output device 155 may output sound signals to the outside ofthe electronic device 101. The sound output device 155 may include, forexample, a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record, and the receivermay be used for an incoming calls. According to an embodiment, thereceiver may be implemented as separate from, or as part of the speaker.

The display device 160 may visually provide information to the outside(e.g., a user) of the electronic device 101. The display device 160 mayinclude, for example, a display, a hologram device, or a projector andcontrol circuitry to control a corresponding one of the display,hologram device, and projector. According to an embodiment, the displaydevice 160 may include touch circuitry adapted to detect a touch, orsensor circuitry (e.g., a pressure sensor) adapted to measure theintensity of force incurred by the touch.

The audio module 170 may convert a sound into an electrical signal andvice versa. According to an embodiment, the audio module 170 may obtainthe sound via the input device 150, or output the sound via the soundoutput device 155 or a headphone of an external electronic device (e.g.,an electronic device 102) directly (e.g., wiredly) or wirelessly coupledwith the electronic device 101.

The sensor module 176 may detect an operational state (e.g., power ortemperature) of the electronic device 101 or an environmental state(e.g., a state of a user) external to the electronic device 101, andthen generate an electrical signal or data value corresponding to thedetected state. According to an embodiment, the sensor module 176 mayinclude, for example, a gesture sensor, a gyro sensor, an atmosphericpressure sensor, a magnetic sensor, an acceleration sensor, a gripsensor, a proximity sensor, a color sensor, an infrared (IR) sensor, abiometric sensor, a temperature sensor, a humidity sensor, or anilluminance sensor.

The interface 177 may support one or more specified protocols to be usedfor the electronic device 101 to be coupled with the external electronicdevice (e.g., the electronic device 102) directly (e.g., wiredly) orwirelessly. According to an embodiment, the interface 177 may include,for example, a high definition multimedia interface (HDMI), a universalserial bus (USB) interface, a secure digital (SD) card interface, or anaudio interface.

A connecting terminal 178 may include a connector via which theelectronic device 101 may be physically connected with the externalelectronic device (e.g., the electronic device 102). According to anembodiment, the connecting terminal 178 may include, for example, a HDMIconnector, a USB connector, a SD card connector, or an audio connector(e.g., a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanicalstimulus (e.g., a vibration or a movement) or electrical stimulus whichmay be recognized by a user via his tactile sensation or kinestheticsensation. According to an embodiment, the haptic module 179 mayinclude, for example, a motor, a piezoelectric element, or an electricstimulator.

The camera module 180 may capture a still image or moving images.According to an embodiment, the camera module 180 may include one ormore lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to theelectronic device 101. According to one embodiment, the power managementmodule 188 may be implemented as at least part of, for example, a powermanagement integrated circuit (PMIC).

The battery 189 may supply power to at least one component of theelectronic device 101. According to an embodiment, the battery 189 mayinclude, for example, a primary cell which is not rechargeable, asecondary cell which is rechargeable, or a fuel cell.

The communication module 190 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 101 and the external electronic device (e.g., theelectronic device 102, the electronic device 104, or the server 108) andperforming communication via the established communication channel. Thecommunication module 190 may include one or more communicationprocessors that are operable independently from the processor 120 (e.g.,the application processor (AP)) and supports a direct (e.g., wired)communication or a wireless communication. According to an embodiment,the communication module 190 may include a wireless communication module192 (e.g., a cellular communication module, a short-range wirelesscommunication module, or a global navigation satellite system (GNSS)communication module) or a wired communication module 194 (e.g., a localarea network (LAN) communication module or a power line communication(PLC) module). A corresponding one of these communication modules maycommunicate with the external electronic device via the first network198 (e.g., a short-range communication network, such as Bluetooth™,wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA))or the second network 199 (e.g., a long-range communication network,such as a cellular network, the Internet, or a computer network (e.g.,LAN or wide area network (WAN)). These various types of communicationmodules may be implemented as a single component (e.g., a single chip),or may be implemented as multi components (e.g., multi chips) separatefrom each other. The wireless communication module 192 may identify andauthenticate the electronic device 101 in a communication network, suchas the first network 198 or the second network 199, using subscriberinformation (e.g., international mobile subscriber identity (IMSI))stored in the subscriber identification module 196.

The antenna module 197 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 101. According to an embodiment, the antenna module197 may include an antenna including a radiating element composed of aconductive material or a conductive pattern formed in or on a substrate(e.g., PCB). According to an embodiment, the antenna module 197 mayinclude a plurality of antennas. In such a case, at least one antennaappropriate for a communication scheme used in the communicationnetwork, such as the first network 198 or the second network 199, may beselected, for example, by the communication module 190 (e.g., thewireless communication module 192) from the plurality of antennas. Thesignal or the power may then be transmitted or received between thecommunication module 190 and the external electronic device via theselected at least one antenna. According to an embodiment, anothercomponent (e.g., a radio frequency integrated circuit (RFIC)) other thanthe radiating element may be additionally formed as part of the antennamodule 197.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (GPIO), serial peripheral interface (SPI), or mobileindustry processor interface (MIPI)).

According to an embodiment, commands or data may be transmitted orreceived between the electronic device 101 and the external electronicdevice 104 via the server 108 coupled with the second network 199. Eachof the electronic devices 102 and 104 may be a device of a same type as,or a different type, from the electronic device 101. According to anembodiment, all or some of operations to be executed at the electronicdevice 101 may be executed at one or more of the external electronicdevices 102, 104, or 108. For example, if the electronic device 101should perform a function or a service automatically, or in response toa request from a user or another device, the electronic device 101,instead of, or in addition to, executing the function or the service,may request the one or more external electronic devices to perform atleast part of the function or the service. The one or more externalelectronic devices receiving the request may perform the at least partof the function or the service requested, or an additional function oran additional service related to the request, and transfer an outcome ofthe performing to the electronic device 101. The electronic device 101may provide the outcome, with or without further processing of theoutcome, as at least part of a reply to the request. To that end, acloud computing, distributed computing, or client-server computingtechnology may be used, for example.

FIG. 2 is a block diagram 200 illustrating the camera module 180according to various embodiments. Referring to FIG. 2, the camera module180 may include a lens assembly 210, a flash 220, an image sensor 230,an image stabilizer 240, memory 250 (e.g., buffer memory), or an imagesignal processor 260. The lens assembly 210 may collect light emitted orreflected from an object whose image is to be taken. The lens assembly210 may include one or more lenses. According to an embodiment, thecamera module 180 may include a plurality of lens assemblies 210. Insuch a case, the camera module 180 may form, for example, a dual camera,a 360-degree camera, or a spherical camera. Some of the plurality oflens assemblies 210 may have the same lens attribute (e.g., view angle,focal length, auto-focusing, f number, or optical zoom), or at least onelens assembly may have one or more lens attributes different from thoseof another lens assembly. The lens assembly 210 may include, forexample, a wide-angle lens or a telephoto lens.

The flash 220 may emit light that is used to reinforce light reflectedfrom an object. According to an embodiment, the flash 220 may includeone or more light emitting diodes (LEDs) (e.g., a red-green-blue (RGB)LED, a white LED, an infrared (IR) LED, or an ultraviolet (UV) LED) or axenon lamp. The image sensor 230 may obtain an image corresponding to anobject by converting light emitted or reflected from the object andtransmitted via the lens assembly 210 into an electrical signal.According to an embodiment, the image sensor 230 may include oneselected from image sensors having different attributes, such as a RGBsensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor, aplurality of image sensors having the same attribute, or a plurality ofimage sensors having different attributes. Each image sensor included inthe image sensor 230 may be implemented using, for example, a chargedcoupled device (CCD) sensor or a complementary metal oxide semiconductor(CMOS) sensor.

The image stabilizer 240 may move the image sensor 230 or at least onelens included in the lens assembly 210 in a particular direction, orcontrol an operational attribute (e.g., adjust the read-out timing) ofthe image sensor 230 in response to the movement of the camera module180 or the electronic device 101 including the camera module 180. Thisallows compensating for at least part of a negative effect (e.g., imageblurring) by the movement on an image being captured. According to anembodiment, the image stabilizer 240 may sense such a movement by thecamera module 180 or the electronic device 101 using a gyro sensor (notshown) or an acceleration sensor (not shown) disposed inside or outsidethe camera module 180. According to an embodiment, the image stabilizer240 may be implemented, for example, as an optical image stabilizer.

The memory 250 may store, at least temporarily, at least part of animage obtained via the image sensor 230 for a subsequent imageprocessing task. For example, if image capturing is delayed due toshutter lag or multiple images are quickly captured, a raw imageobtained (e.g., a Bayer-patterned image, a high-resolution image) may bestored in the memory 250, and its corresponding copy image (e.g., alow-resolution image) may be previewed via the display device 160.Thereafter, if a specified condition is met (e.g., by a user's input orsystem command), at least part of the raw image stored in the memory 250may be obtained and processed, for example, by the image signalprocessor 260. According to an embodiment, the memory 250 may beconfigured as at least part of the memory 130 or as a separate memorythat is operated independently from the memory 130.

The image signal processor 260 may perform one or more image processingwith respect to an image obtained via the image sensor 230 or an imagestored in the memory 250. The one or more image processing may include,for example, depth map generation, three-dimensional (3D) modeling,panorama generation, feature point extraction, image synthesizing, orimage compensation (e.g., noise reduction, resolution adjustment,brightness adjustment, blurring, sharpening, or softening). Additionallyor alternatively, the image signal processor 260 may perform control(e.g., exposure time control or read-out timing control) with respect toat least one (e.g., the image sensor 230) of the components included inthe camera module 180. An image processed by the image signal processor260 may be stored back in the memory 250 for further processing, or maybe provided to an external component (e.g., the memory 130, the displaydevice 160, the electronic device 102, the electronic device 104, or theserver 108) outside the camera module 180. According to an embodiment,the image signal processor 260 may be configured as at least part of theprocessor 120, or as a separate processor that is operated independentlyfrom the processor 120. If the image signal processor 260 is configuredas a separate processor from the processor 120, at least one imageprocessed by the image signal processor 260 may be displayed, by theprocessor 120, via the display device 160 as it is or after beingfurther processed.

According to an embodiment, the electronic device 101 may include aplurality of camera modules 180 having different attributes orfunctions. In such a case, at least one of the plurality of cameramodules 180 may form, for example, a wide-angle camera and at leastanother of the plurality of camera modules 180 may form a telephotocamera. Similarly, at least one of the plurality of camera modules 180may form, for example, a front camera and at least another of theplurality of camera modules 180 may form a rear camera.

Herein is disclosed an electronic device and a method for securingpersonal information included in an image. An aspect of an embodimentdisclosed herein is to prevent the unintended leakage of the user'spersonal information that may be included in the image.

According to certain embodiments, an electronic device and a method forsecuring personal information included in an image can prevent personalinformation included in the image from being leaked to the public.

According to certain embodiments, in an electronic device and a methodfor securing personal information included in an image, the user'spersonal information included in the image may be displayed only on theuser's electronic device.

FIG. 3 is a block diagram illustrating operations of an electronicdevice and an external electronic device according to an embodiment.

In an embodiment, the electronic device 101 may include an image sensor321, an image signal processor (ISP) 323, and a memory 325. An externalelectronic device 300 (e.g. the electronic device 104 or the server 108)may include a recognition module 331, an ISP 333, and a storage 335. Therecognition module 331 may be a logical module and may be implementedusing a processor of the external electronic device 300 together withsoftware. The ISP 333 may also be implemented using the processor of theexternal electronic device 300. In that example, the processor of theexternal electronic device 300 may perform both recognition and imageprocessing. Although not illustrated, the electronic device 101 mayinclude a communication module (e.g. the communication module 190 inFIG. 1) which can transmit or receive data to or from the externalelectronic device 300. The external electronic device 300 may include acommunication module which can transmit or receive data to or from theelectronic device 101. According to an embodiment, the electronic device101 may include a recognition module 327. The recognition module 327disposed in the electronic device 101 may be configured to performfunctions identical to some of functions of the recognition module 331.For example, the recognition module 327 may be hardware or softwareconfigured to recognize a face in an image, and may be able to moresimply and quickly recognize the face than the external electronicdevice 300 (e.g. the server) because the recognition is done locallyrather than requiring the image to be transmitted over a network.

In an embodiment, the image sensor 321 (e.g. the image sensor 230 inFIG. 2) may acquire an image of an external object, and may generate araw image 322 corresponding thereto. The raw image 322 may beimplemented in various formats, such as Bayer format, color filter array(CFA) pattern, layer structure generated by sensing three colors foreach pixel, format generated by different parallax acquired by eachpixel, and the like. The image sensor 321 may transfer the raw image 322to the ISP 323 (e.g. the image signal processor 260 in FIG. 2) and/orthe recognition module 327.

In an embodiment, the image sensor 321 may generate a small raw image326. The image sensor 321 may generate the small raw image 326 byreducing the raw image 322 by, for example, reducing its resolution. Forexample, the image sensor 321 may generate the small raw image 326 fromthe raw image 322 by using various down-scaling methods or down-samplingmethods. The data volume of the small raw image 326 may be smaller thanthat of the raw image 322 due to the image sensor 321 adjusting theresolution of the raw image 322, filtering the raw image 322 through oneor more frequency bands, and/or selecting one or more bit-plane levelsof the raw image 322. For example, the image sensor 321 may generate thesmall raw image 326 by extracting a low-frequency band from the rawimage 322. In another example, the image sensor 321 may generate thesmall raw image 326 by selecting some from among a multiple bit-planelevels of the raw image 322. The image sensor 321 may transmit the smallraw image 326 to the external electronic device 300 through thecommunication module. The small raw image 326 may include at least apart of the raw image 322, and may be a smaller data file than the rawimage 322. If the small raw image 326, instead of the raw image 322, istransmitted to the external electronic device, volume of datatransmission is reduced, and thus the image can be more quicklytransmitted to the external electronic device 300. In anotherembodiment, a processor (e.g. the processor 120) of the electronicdevice 101, instead of the image sensor 321, may generate the small rawimage 326, and may transmit the generated small raw image 326 to theexternal electronic device 300 through the communication module. Theprocessor 120 may include a microprocessor or any suitable type ofprocessing circuitry, such as one or more general-purpose processors(e.g., ARM-based processors), a Digital Signal Processor (DSP), aProgrammable Logic Device (PLD), an Application-Specific IntegratedCircuit (ASIC), a Field-Programmable Gate Array (FPGA), a GraphicalProcessing Unit (GPU), a video card controller, etc. In addition, itwould be recognized that when a general purpose computer accesses codefor implementing the processing shown herein, the execution of the codetransforms the general purpose computer into a special purpose computerfor executing the processing shown herein. Certain of the functions andsteps provided in the Figures may be implemented in hardware, softwareor a combination of both and may be performed in whole or in part withinthe programmed instructions of a computer. No claim element herein is tobe construed under the provisions of 35 U.S.C. § 112(f), unless theelement is expressly recited using the phrase “means for.” In addition,an artisan understands and appreciates that a “processor” or“microprocessor” may be hardware in the claimed disclosure. Under thebroadest reasonable interpretation, the appended claims are statutorysubject matter in compliance with 35 U.S.C. § 101.

In an embodiment, the image sensor 321 may transmit a compressed rawimage 322 to the ISP 323, the external electronic device 300, or therecognition module 327. The image sensor 321 may compress the raw image322 and may store the compressed raw image 322 in an internal memory ofthe image sensor 321.

In an embodiment, the recognition module 331 of the external electronicdevice 300 may acquire the small raw image 326 through the communicationmodule, and may segment the small raw image 326 into at least one imagesegment. The recognition module 331 may recognize the at least one imagesegment that resulted from the segmentation. For example, therecognition module 331 may segment the small raw image 326, and mayidentify at least one image segment from the small raw image 326 thatresulted from the segmentation. The recognition module 331 may recognizethe image segment by applying an object recognition algorithm or a textrecognition algorithm to the image segment. Thus, the recognition module331 of the external electronic device 300 may recognize at least oneimage segment by using various recognition algorithms, such as a machinelearning or deep learning algorithm. For example, the recognition module331 of the external electronic device 300 may acquire informationassociated with an image segment in which pixel coordinates (100, 101),(100, 102), (102, 102), and (102, 103) represent “person's teeth.” Thepixel coordinates may correspond to pixel coordinates of the raw image322. The recognition module 331 may acquire, for example, classificationinformation indicating that the small raw image 326 is classified as acategory in which “a person is on a street.” The recognition module 331may acquire classification information by using the recognition result,or may acquire classification information by color distribution, etc. inthe small raw image 326 without employing recognition. The recognitionmodule 331 may generate correction segment information 332 including atleast one of the classification information or the informationassociated with the at least one image segment acquired through theabove-described process. The recognition module 331 may transmit thecorrection segment information 332 to the electronic device 101. The ISP323 may correct the raw image 322 by using the correction segmentinformation 332, and thus may generate a corrected image 324. Thecorrected image 324 may have, for example, a YUV format. The correctedimage 324 may be stored in the memory 325. Alternatively, the correctedimage 324 may be compressed by, for example, JPEG method, and thecompressed image may be stored in the memory 325. According to anembodiment, the correction segment information may be generated by therecognition module 327 of the electronic device 101 and may then betransferred to the ISP 323.

In an embodiment, the raw image 322 provided from the image sensor 321may be transmitted to the external electronic device 300, regardless ofwhether the small raw image 326 is transmitted. The external electronicdevice 300 may generate another correction segment information by usingthe raw image 322. By using the raw image 322, which is larger than thesmall raw image 326, the external electronic device 300 (e.g. the ISP333 of the external electronic device 300) may generate correctionsegment information based on the raw image 322 in addition to thecorrection segment information generated from the small raw image 326,and may designate the same as extended correction segment information.Since the raw image 322 may include more information than the small rawimage 326, the external electronic device 300 may generate more detailedcorrection segment information from the raw image 322. In oneembodiment, the external electronic device 300 (the ISP 333 of theexternal electronic device 300) may only generate extended correctionsegment information from the raw image 322. Alternatively, the externalelectronic device 300 (e.g. the ISP 333 of the external electronicdevice 300) may generate extended correction segment information byusing the raw image 322 as well as correction segment informationgenerated from the small raw image 326.

In an embodiment, since the raw image 322 is larger than the small rawimage 326, the small raw image 326 may be first transmitted to theexternal electronic device 300, followed by transmission of the rawimage 322 to the external electronic device 300. For example, the rawimage 322 may be transmitted to the external electronic device 300 whilethe ISP 323 corrects the raw image 322 using correction segmentinformation generated from the small raw image 326. The raw image 322may be uploaded to the external electronic device 300 as generated bythe image sensor 321. Alternatively, a pre-processed version of the rawimage 322, which has undergone lens distortion correction and noiseremoval, may be uploaded. The pre-processing may be performed by theexternal electronic device 300. The external electronic device 300 mayperform demosaic processing or image format modification, or otherpre-processing to improve the image recognition done by the recognitionmodule 331. The ISP 333 of the external electronic device 300 maycorrect the received raw image 322. The external electronic device 300may correct the raw image 322 by using the correction segmentinformation 332 and/or the extended correction segment information. Asthe raw image 322 may be of higher quality than the small raw image 326,the ISP 333 of the external device 300 may acquire more detailed segmentinformation from the higher quality image. The ISP 333 may generate theextended correction segment information by using the generatedcorrection segment information and the raw image 322 together. The ISP333 may generate a high quality image 334 by correcting the raw image322 by using the extended correction segment information. The highquality image 334 may be stored in the storage 335 of the externalelectronic device 300 and may be downloaded to the electronic device101.

FIG. 4 is a block diagram of electronic devices according to anembodiment.

A first electronic device 470 (e.g. the electronic device 102 or theserver 108) according to an embodiment may be implemented as a cloudserver. The first electronic device 470 may perform network management,cloud service management relating to services and rights to be provided,and storage management, with respect to servers constituting a cloudsystem and with respect to electronic devices (e.g. a second electronicdevice 400) connectable to the cloud system.

The first electronic device 470 according to an embodiment may include aprocessor 471, a database 478, a raw image storage 479, and an imagestorage 480. The processor 471 according to an embodiment may include apre-processing module 474, an image recognition module 473, an ISP 476,an encoder 475, a monitoring module 472, a test target acquisitionmodule 481, an image comparison module 482, and a correction targetdetermination module 483. According to an embodiment, the ISP 476 may beimplemented as separate hardware.

According to an embodiment, the pre-processing module 474, the imagerecognition module 473, the encoder 475, the monitoring module 472, thetest target acquisition module 481, the image comparison module 482, andthe correction target determination module 483 may be logical modules.Thus, operations performed thereby may be performed by the processor 471or a processor (e.g. a first processor 430 or a second processor 412) ofthe second electronic device 400.

According to an embodiment, at least one of the pre-processing module474, the image recognition module 473, the ISP 476, the encoder 475, themonitoring module 472, the test target acquisition module 481, the imagecomparison module 482, or the correction target determination module 483may be implemented as a separate hardware module in the processor 471 ofthe first electronic device 470 or a separate hardware module in theprocessor (e.g. the first processor 430 or the second processor 412) ofthe second electronic device 400.

The pre-processing module 474 according to an embodiment may pre-processan image (e.g. raw image or small raw image) received from the secondelectronic device 400 and may transmit the pre-processed image to theimage recognition module 473, the raw image storage 479, or the ISP 476.According to an embodiment, the pre-processing may include decoding theimage file received from the second electronic device 400 to generate araw image.

The image recognition module 473 (e.g. the recognition module 331 inFIG. 3) according to an embodiment may receive an image (e.g. raw imageor small raw image) from the second electronic device 400 through acommunication module of the first electronic device 470 or viapre-processing module 474 thereof.

The image recognition module 473 according to an embodiment mayrecognize the received image by using various algorithms, such as (1) anobject recognition algorithm, 2) a text recognition algorithm, or 3) arecognition algorithm acquired by applying machine learning or deeplearning to an image storage (e.g. the raw image storage 479 and/or theimage storage 480)). The image recognition module 473 may store therecognized result in a database (e.g. the raw image storage 479) inassociation with to the received image and/or may transmit therecognized result to the second electronic device 400. For example, theimage recognition module 473 may recognize at least one object (e.g. aperson, the sky, a cloud, a cat, a puppy, etc.) in the received image.The object may be recognized in an image segment within the image.According to an embodiment, the image recognition module 473 mayrecognize an object in an image corresponding to user information. Forexample, the user information may be acquired from an image (e.g. apicture of family or acquaintance, or a house picture) stored in thedatabase 478. The recognized result may include position information(e.g. pixel coordinates) of an image segment and identificationinformation of an object in the image segment. Further the recognizedresult may be used when the ISP 476 processes an image and when the testtarget acquisition module 481 selects a test target.

The image recognition module 473 according to an embodiment may storeidentification information of an object (or objects) recognized in thereceived image as category information of the received image, which maybe stored in a database (e.g. the raw image storage 479) in associatedwith the category information. For example, the image recognition module473 may store the identification information as tag information of areceived image and store the same in a database. Alternatively, theimage recognition module 473 may transmit the identification informationto the second electronic device 400. The category information mayinclude multiple categories which are hierarchically classified. Forexample, if a “cat” is recognized as an object in an image segment, theimage recognition module 473 may configure identification informationindicating an “animal” as a superordinate category of the correspondingimage and may configure identification information indicating the “cat”as a subordinate category of the corresponding image. According to anembodiment, the first electronic device 470 may receive categoryinformation corresponding to an image (e.g. raw image or small rawimage) from the second electronic device 400, and may store the categoryinformation in the database in relation to the corresponding image.

The ISP 476 according to an embodiment may perform image processing(e.g. correction) on the received image by adjusting various values(e.g. brightness, shade, color, saturation, color temperature,sharpness, color harmony, vividness, contrast, or hue saturation value(HSV)) of the received image. The image processing may be performedbased on recipe information (e.g. image segment, layer, vector, scenecategory, etc.) acquired through image recognition. For example, the ISP476 may recognize an image segment, in which an object is positioned, ofan image as a correction target, and may correct the image segment. TheISP 476 may receive additional information (e.g. feature vectorindicating a feature of an object or a part thereof (e.g. a hair))corresponding to the recipe information from the database 478, and mayuse the additional information for image processing. The processed imagemay be transmitted to the second electronic device 400 or the encoder475, or may be stored in a database (e.g. the image storage 480). Theimage processing may include functions, such as white balance, coloradjustment, noise reduction, sharpness effect, and detail enhancement.The functions may be performed for each image segment on the basis ofthe recipe information.

The encoder 475 according to an embodiment may generate an image file(e.g. JPEG, MPEG, 360-degree panorama, etc.) by encoding the correctedimage processed by the ISP 476 or a raw image stored in the raw imagestorage 479. The image file generated by the encoder 475 may betransmitted to the second electronic device 400 through thecommunication module of the first electronic device 470, or may bestored in the database (e.g. the image storage 480).

The monitoring module 472 according to an embodiment may determine thatthe ISP 476, which is configured to process images in the firstelectronic device 470, has been changed from a first ISP 476 a to asecond ISP 476 b. For example, the second ISP 476 b may perform anupgraded correction method from that of the first ISP 476 a.

In response to the determination made by the monitoring module 472, thetest target acquisition module 481 according to an embodiment mayacquire some of the images stored in the database (e.g. the raw imagestorage 479 or the image storage 480) as test targets for imageprocessing. According to one embodiment, if the test targets areacquired from the raw image storage 479, image processing may beperformed by the correction method of the first ISP 476 a, to which amodification (e.g. replacement or upgrade) is not applied, and also thecorrection method of the second ISP 476 b, to which modification hasbeen applied. According to one embodiment, if the test targets areacquired from the image storage 480, image processing may be performedby on the second ISP 476 b.

The test target acquisition module 481 according to an embodiment mayacquire an equal number of test targets for one or more categories. Forexample, the test target acquisition module 481 may acquire the samenumber of images classified to the “puppy” category as the number ofimages classified as the “cat” category.

In an embodiment, each of the first ISP 476 a and the second ISP 476 bmay correct the test targets acquired by the test target acquisitionmodule 481, i.e. by adjusting various values (e.g. brightness, shade,color, saturation, color temperature, sharpness, color harmony,vividness, contrast, or HSV) of each the test targets. According to anembodiment, if a test target is a compressed image acquired from theimage storage 480, the image may be decompressed first and thencorrected.

The image comparison module 482 according to an embodiment maycalculate, for each of the test targets acquired from the databases(e.g. the raw image storage 479), the difference between the valuesobtained through image correction by the first ISP 476 a and obtainedthrough image correction by the second ISP 476 b. For example, the imagecomparison module 482 may select, as a comparison target, at least onefrom among brightness, shade, color, saturation, color temperature,sharpness, color harmony, vividness, contrast, or HSV on the basis of atuning policy (e.g., effect) that is expected as the correction methodis changed. For example, if the effect expected by the change of thecorrection method is an increase in contrast, the image comparisonmodule 482 may calculate, for each of the test targets, the differencebetween the contrast values obtained by the image correction of thefirst ISP 476 a and the image correction of the second ISP 476 b. If theexpected effect is an increase in the vividness of an image segmentwhere the HSV has about 30 to 60 degrees, the image comparison module482 may calculate, for each of the test targets, the difference betweenthe color values obtained by image correction of the first ISP 476 a andthe image correction of the second ISP 476 b.

The image comparison module 482 according to an embodiment maycalculate, for each of the test targets acquired from the database (e.g.the image storage 480), the difference between a value of the imagesbefore and after the correction by the second ISP 476 b.

The image comparison module 482 according to an embodiment may classifythe test targets into two groups on the basis of the comparison result.For example, the image comparison module 482 may classify the testtargets into a superordinate group having a difference in valueexceeding a predetermined threshold and a subordinate group having adifference in value below the threshold. Here, the difference in valuemay refer to 1) the difference between values in images obtained byimage correction of the first ISP 476 a and image correction of thesecond ISP 476 b, or 2) the difference between values in images beforeand after correction by the second ISP 476 b.

The correction target determination module 483 according to anembodiment may determine a correction target on the basis of thecomparison result acquired by the image comparison module 482. Here, thecorrection target may be selected from among images stored in thedatabases (e.g. the raw image storage 479 or the image storage 480). Thecorrection target may undergo image processing by a correction method(e.g., the second ISP 476 b) to which a modification (e.g. replacementor upgrade) has been applied, and then may be stored in the database(e.g. the image storage 480) and/or may be transmitted to the secondelectronic device 400. According to one embodiment, the correctiontarget determination module 483 may recognize the subordinate group as agroup having the correction effect which is not as expected, andaccordingly, may identify a category of images belonging to thesubordinate group, and may determine that images corresponding to thecategory of the subordinate group are not subject to be corrected. Thecorrection target determination module 483 may select at least one testtarget from the superordinate group and may determine imagescorresponding to the category of the selected test target as correctiontargets among the images stored in the databases (e.g. the raw imagestorage 479 or the image storage 480).

According to an embodiment, category information used to determine thecorrection target may include information acquired by the imagerecognition module 473 and then stored in the database (e.g. the rawimage storage 479 or the image storage 480) in association with thecorresponding image. Further, the category information may includeinformation which is received from an external device (e.g. the secondelectronic device 400) and stored in the database as metadata of thecorresponding image together with the corresponding image.

In an embodiment, according to the comparison result acquired by theimage comparison module 482, a category may be commonly associated withthe test targets of the superordinate group or test targets of thesubordinate group. For example, a first test target having a “cat” maybelong to the superordinate group, while a second test target alsohaving a “cat” may belong to the subordinate group. Thus, images thatfall into both superordinate and subordinate groups may have lesscorrection effects than expected. Therefore, the correction targetdetermination module 483 may exclude images having a “cat” fromcorrection targets.

As described above, according to one embodiment, the correction targetdetermination module 483 may determine, as a correction target, a testtarget corresponding to category information which corresponds to afirst level and does not correspond to a second level. The categorycorresponding to the first level may be a category of a test targetbelonging to a superordinate group according to the classification, andthe category corresponding to the second level may be a category of atest target belonging to a subordinate group according to theclassification.

In an embodiment, according to the comparison result acquired by theimage comparison module 482, some metadata may be commonly associatedwith test targets belonging to the superordinate group and test targetsbelonging to the subordinate group. For example, although having thesame ISO sensitivity, a first test target may belong to thesuperordinate group, while a second test target may belong to thesubordinate group. In addition, specific recipe information (e.g. ascene category) may be commonly associated with a test target belongingto the superordinate group and a test target belonging to thesubordinate group. As described above, images corresponding to themetadata or recipe information commonly associated with thesuperordinate group and the subordinate group may have less correctioneffect than expected. Accordingly, the correction target determinationmodule 483 may exclude, from the correction targets, images havingmetadata or recipe information associated with the superordinate groupand the subordinate group among the images stored in the databases (e.g.the raw image storage 479 or the image storage 480).

According to one embodiment, the correction target determination module483 may determine a correction target on the basis of at least one ofmetadata or recipe information, as well as on the basis of thecomparison result acquired by the operation of the image comparisonmodule 482. For example, the correction target determination module 483may identify metadata of test targets belonging to the superordinategroup (hereinafter, referred to as an “upper metadata set”), andmetadata of test targets belonging to the subordinate group (hereinafterreferred to as a “lower metadata set”). The correction targetdetermination module 483 may compare the upper metadata set and thelower metadata set to identify metadata (hereinafter, referred to as a“duplicate metadata set”) included in both the upper metadata set andthe lower metadata set. The correction target determination module 483may determine, as the correction targets, images that corresponds to oneor more metadata of the upper metadata set and does not correspond totwo or more metadata of the duplicate metadata set. For example, if theupper metadata set includes a first ISO sensitivity value, a second ISOsensitivity value, a first brightness value, a second brightness value,a first sharpness value, and a second sharpness value, and the first ISOsensitivity value, the first brightness value, and the first sharpnessvalue are determined as duplicate metadata, images having the first ISOsensitivity value, the second brightness value, and the second sharpnessvalue may be determined as correction targets. Images having the firstISO sensitivity value, the first brightness value, and the secondsharpness value may be excluded from the correction targets.

According to an embodiment, the metadata may be received from anexternal device (e.g. the electronic device 400) together with an image,and may be stored in the database (e.g. the raw image storage 479 or theimage storage 480) in association with the image. For example, themetadata may be information acquired by an image sensor (e.g. an imagesensor 411), and may include a focal length, an auto focus area,information (orientation) related to left and right rotation duringphotographing, auto white balance (AWB), color space, exposure time,aperture-related information (e.g. F-number, F-stop), exposure program(e.g. auto, aperture priority, shutter priority, manual, etc.), ISOspeed ratings (ISO), date of image capturing (data time original), etc.The metadata may also include information, such as illuminance at animage-capturing place or time stamp, sensed by a sensor or clock otherthan the image sensor.

According to an embodiment, the recipe information may includeinformation (e.g. image segment, layer, vector, scene category, etc.)acquired by the image recognition module 473 by recognizing an image(e.g. small raw image) received from an external device (e.g. theelectronic device 400).

According to an embodiment, the correction target determination module483 may determine correction targets on the basis of a tuning policy(e.g. an effect). For example, when images are corrected by a correctionmethod (e.g., the second ISP 476 b) to which modification (e.g.replacement or upgrade) has been applied, if the expected effect is anincrease in brightness, the correction target determination module 483may apply a weight to the difference value in brightness (e.g. may add apredetermined value to the difference value) so as to preferentiallyclassify the test targets having brightness increased by the correctioninto the superordinate group.

The second electronic device 400 (e.g. the electronic device 101)according to an embodiment may include a camera module 410, a display420, a first processor 430, a first memory 440, an input device 450, anda sound output device 460. The camera module 410 according to anembodiment may include an image sensor 411, a second processor 412, anda second memory 418. The second processor 412 according to an embodimentmay include a raw image processing module 413, an ISP 416, and anencoder 417. At least some of operations performed by the secondprocessor 412 may be performed by the first processor 430. The raw imageprocessing module 413, the ISP 416, and the encoder 417 may be logicalmodules, and thus, operations performed thereby may be performed by thesecond processor 412 (e.g. the processor 120). In another embodiment, atleast one of the raw image processing module 413, the ISP 416, and theencoder 417 may be implemented as separate hardware modules within thesecond processor 412.

Although not illustrated, the second electronic device 400 may include acommunication module (e.g. the communication module 190 in FIG. 1) fordata communication with the first electronic device 470, and the firstelectronic device 470 may also include a communication module for datacommunication with the second electronic device 400.

In an embodiment, the image sensor 411 (e.g. the image sensor 230 inFIG. 2 or the image sensor 321 in FIG. 3) may acquire various raw imagesof a subject. The image sensor 411 may acquire various types of rawimages according to color filter array (CFA) pattern. The image sensor411 may acquire a raw image having a dual pixel (DP) structure (or dualphotodiode (2PD) structure) in which a pixel includes different parallax(or phase difference) information. The image sensor 411 may includemultiple image sensors having identical or different characteristics(e.g. dual sensors (e.g. RGB+RGB, RGB+Mono, or Wide+Tele, etc.), or anarray sensor (e.g. two or more sensors), etc.). The multiple imagesensors may be used to acquire one or more raw images. The acquired rawimages may be stored in the second memory 418 (e.g. DRAM) as is or afteradditional processing.

The raw image acquired according to an embodiment may be configured invarious formats (e.g. Bayer format, etc.). The raw image may berepresented by one of red (R), green (G), and blue (B) per pixel, andmay be represented by a bit-depth of 8 to 16 bits. Various color filterarray (CFA) patterns may be applied to the raw image. The raw image mayhave a layer structure including information on various colors (e.g.multiple colors of R, and B) for each pixel. According to variousconfigurations of the image sensor, the raw image may include not onlycolor information but also parallax (phase difference) information.Information (e.g. time, location, illuminance, etc.) related tocapturing an image may be generated as metadata and stored inassociation with the raw image. For example, the second processor 412may acquire metadata about the raw image using the image sensor 411. Themetadata that can be acquired through the image sensor 411 includes, forexample, focal length, auto focus area, information related to left andright rotation during image capturing (orientation), color space, andexposure time. In addition, the metadata may include locationinformation of an image, which may be acquired through a sensor (e.g. aGPS sensor) different from the image sensor.

In an embodiment, the raw image processing module 413 may performvarious processing on the raw image acquired from the image sensor 411.For example, the raw image processing module 413 may perform lensdistortion correction or noise removal on the raw image.

The raw image processing module 413 according to an embodiment mayinclude a small raw image generation module 414 and a compression module415. The small raw image generation module 414 may generate a small rawimage from a raw image by using various down-scaling methods (e.g.reducing the size or reducing the resolution) or down-sampling methods(e.g. taking only one or some of a series of sampled samples). Thecompression module 415 may compress the raw image or the small raw imageby using various compression algorithms, and may store the compressedraw image or the compressed small raw image in the second memory 418.The small raw image may be temporarily or persistently stored in thesecond memory 418. The communication module (not shown) (e.g. thecommunication module 190) of the second electronic device 400 maytransmit the small raw image stored in the second memory 418 to thefirst electronic device 470.

In an embodiment, the ISP 416 (e.g. the image signal processor 260 inFIG. 2 or the ISP 323 in FIG. 3) may perform image processing on a rawimage stored in second memory 418. For example, the ISP 416 may process(e.g. correct) the raw image by using recipe information (e.g. imagesegment, layer, vector, scene category, etc.) acquired from the firstelectronic device 470 through the communication module in various ways.According to one embodiment, the ISP 416 may transmit the raw image orthe small raw image to the first electronic device 470 through thecommunication module so that the first electronic device 470 generatesrecipe information for image processing. The ISP 416 may process the rawimage by using the recipe information acquired from the first electronicdevice 470 through the communication module. The ISP 416 may compressthe processed raw image into, for example, JPEG and may store the samein the first memory 440.

In an embodiment, the ISP 416 that performs image processing in thesecond electronic device 400 may be changed from a first ISP 416 a to asecond ISP 416 b. For example, the second ISP 416 b may have acorrection method in which at least a part of the correction method ofthe first ISP 416 a has been upgraded.

In an embodiment, the encoder 417 may encode the raw image processed bythe ISP 416 to generate an image file (e.g. JPEG, MPEG, 360 degreepanorama, etc.), and may store the image file in the first memory 440.

In an embodiment, the first processor 430 (e.g. the processor 120 inFIG. 1) may be electrically connected to the camera module 410, thedisplay 420, the first memory 440, the input device 450, the soundoutput device 460, and the communication module (not shown) so as tocontrol at least one thereof. Further, the first processor 430 mayperform various data processing and operations.

According to an embodiment, the first processor 430 may update an imagestored in the first memory 440 in response to the fact that the ISP 416has changed from the first ISP 416 a to the second ISP 416 b. Forexample, the first processor 430 may perform functions identical to atleast some functions of the processor 471 of the first electronic device470 (e.g. at least one among the test target acquisition module 481, theimage comparison module 482, and the correction target determinationmodule 483).

According to an embodiment, an electronic device (e.g. the electronicdevice 101 in FIG. 1 or the second electronic device 400 in FIG. 4) mayinclude: a camera (e.g. the camera module 180 in FIG. 1) including animage sensor (e.g. the camera module 180 in FIG. 1, the image sensor 230in FIG. 2, the image sensor 321 in FIG. 3, or the image sensor 411 inFIG. 4); a communication module (e.g. the communication module 190 inFIG. 1); a memory (e.g. the memory 130 in FIG. 1, the memory 250 in FIG.2, or the memory 325 in FIG. 3); and a processor (e.g. the processor 120in FIG. 1, the image signal processor 260 in FIG. 2, the ISP 323 in FIG.3, or the first processor 430 or the second processor 412 in FIG. 4)operationally connected to the camera, the communication module, and thememory. The memory may store instructions that, when executed, cause theprocessor to: acquire a first image including an external object byusing the camera; acquire a second image including less information thanthe first image by using at least a part of the first image; transmitthe second image to an external electronic device (e.g. the electronicdevice 104 or the server 108 in FIG. 1, the external electronic device300 in FIG. 3, or the first electronic device 470 in FIG. 4) by usingthe communication module; receive security information corresponding tothe second image from the external electronic device by using thecommunication module; and secure the first image, at least partiallybased on the security information.

According to an embodiment, the security information corresponding tothe second image may relate to at least one biometric informationsegment of the external object.

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to divide, at least partially based onthe security information, the first image into a first security levelsegment and a second security level segment.

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to: generate a security key and storethe security key in the memory (e.g. the memory 130 in FIG. 1, thememory 250 in FIG. 2, or the memory 325 in FIG. 3); acquire a thirdimage in which the second security level segment of the first image hasbeen encoded based on the security key; and transmit the third image tothe external electronic device (e.g. the electronic device 104 or theserver 108 in FIG. 1, the external electronic device 300 in FIG. 3, orthe first electronic device 470 in FIG. 4).

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to: receive a fourth image, in which thefirst security level segment of the third image has been at leastpartially corrected, from the external electronic device (e.g. theelectronic device 104 or the server 108 in FIG. 1, the externalelectronic device 300 in FIG. 3, or the first electronic device 470 inFIG. 4); and acquire a fifth image in which the fourth image has beendecoded using the security key stored in the memory (e.g. the memory 130in FIG. 1, the memory 250 in FIG. 2, or the memory 325 in FIG. 3).

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to acquire the third image bymosaicking, masking, or scrambling the second security level segment ofthe first image.

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to: extract the first security levelsegment from the first image and generate the third image including thefirst security level segment; and extract the second security levelsegment from the first image and generate a security segment image basedon the second security level segment.

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to: transmit the third image to theexternal electronic device; and store the security segment image in thememory (e.g. the memory 130 in FIG. 1, the memory 250 in FIG. 2, or thememory 325 in FIG. 3).

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to: receive the fourth image, in whichthe third image has been at least partially corrected, from the externalelectronic device (e.g. the electronic device 104 or the server 108 inFIG. 1, the external electronic device 300 in FIG. 3, or the firstelectronic device 470 in FIG. 4); and acquire the fifth image bysynthesizing the fourth image and the security segment image.

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to: provide, based on the receivedsecurity information, a user with an image indicating the first securitylevel segment and the second security level segment; and separate thefirst security level segment and the second security level segment fromeach other on the basis of a user input selecting a part or an entiretyof the first security level segment or the second security levelsegment.

According to an embodiment, the instructions may further cause theprocessor (e.g. the processor 120 in FIG. 1, the image signal processor260 in FIG. 2, the ISP 323 in FIG. 3, or the first processor 430 or thesecond processor 412 in FIG. 4) to: receive another user inputdesignating another segment other than the first security level segmentand the second security level segment in the provided image; anddetermine the designated another segment as the second security levelsegment.

FIG. 5 is a flowchart illustrating a method for securing personalinformation included in an image in an electronic device according to anembodiment.

Referring to FIG. 5, in operation 510, according to an embodiment, theprocessor 120 of the electronic device 101 (e.g. the second electronicdevice 400 in FIG. 4) may acquire a first image including an externalobject by using a camera including an image sensor (e.g. the cameramodule 180 in FIG. 1, the image sensor 230 in FIG. 2, the image sensor321 in FIG. 3, or the image sensor 411 in FIG. 4). The first image maybe a raw image (e.g. the raw image 322 in FIG. 3) and/or an image (e.g.a JPEG image) obtained by processing or correcting the raw image.

According to an embodiment, the first image may be in various formats(e.g. Bayer format, etc.). The first image may be represented by one ofred (R), green (G), and blue (B) per pixel, and may be represented by abit-depth of 8 to 16 bits. Various color filter array (CFA) patterns maybe applied to the first image. The first image may have a layerstructure including information on various colors (e.g. multiple colorsof R, G, and B) for each pixel. According to various configurations ofthe image sensor, the first image may include not only color information(e.g. RGB) but also phase difference information. Information (e.g.time, location, illuminance, etc.) related to capturing of the image maybe generated as metadata and stored in association with the first image.

In operation 520, according to an embodiment, the processor 120 of theelectronic device 101 may acquire a second image including lessinformation than the first image by using at least a part of the firstimage. The second image may be a small image (the small raw image 326 inFIG. 3) related to the raw image and/or a small image of a processedimage (e.g. a JPEG image).

According to an embodiment, the processor 120 may generate the secondimage by reducing the data in the first image. For example, theprocessor 120 may generate the second image from the first image byusing various down-scaling methods or down-sampling methods. The datavolume of the second image may be smaller than that of the first imagedue to the processor 120 adjusting the resolution of the first image,filtering the first image through one or more frequency bands, and/orselecting one or more bit-plane levels of the fir image. The processor120 may generate the second image by extracting a low-frequency bandfrom the first image. The processor 120 may generate the second image byselecting some from among the multiple bit-plane levels of the firstimage.

In operation 530, according to an embodiment, the processor 120 of theelectronic device 101 may transmit the second image to the externalelectronic device 300 (e.g. the electronic device 104 or the server 108in FIG. 1 or the first electronic device 470 in FIG. 4) through thecommunication module (e.g. the communication module 190 in FIG. 1).

According to an embodiment, the processor 120 may transmit the secondimage having a smaller capacity than the first image to the externalelectronic device 300 such that the external electronic device 300generates security recipe information for securing the first or secondimage. According to one embodiment, the processor 120 may requestsecurity recipe information from the external electronic device 300 soas to divide the second image into a first security level segment and asecond security level segment requiring higher security than the firstsecurity level segment.

According to an embodiment, the security recipe information may be usedto divide the second image into multiple security level segments (e.g.three security level segments). In this instance, multiple servers mayprocess the second image for multiple security levels. For example, afirst server may be configured to process a first security level segmentincluded in the second image, and a second server may be configured toprocess a second security level segment. In another example, a thirdsecurity level segment may be processed by the electronic device 101.

In operation 540, according to an embodiment, the processor 120 of theelectronic device 101 may receive security information included in thesecond image from the external electronic device 300 by using thecommunication module (e.g. the communication module 190 in FIG. 1).

According to an embodiment, the external electronic device 300 may usethe second image to recognize at least one image segment. For example,the external electronic device 300 may determine at least one imagesegment (e.g. a security segment) that includes personal information.

According to an embodiment, instead or in addition to determiningpersonal information (e.g. fingerprint information, iris information, orface information), the external electronic device 300 may apply anobject recognition algorithm or a text recognition algorithm to thesecond image so as to identify an object (e.g. a person) or recognize aspecific segment of the object (e.g. “the person's face,” “the person'seye,” or “the person's hand”). For example, the external electronicdevice 300 may identify an object (e.g. a driver's license) in thesecond image, but may not determine personal information (e.g. licensenumber) included in the object.

According to an embodiment, the external electronic device 300 mayinclude at least one image segment (e.g. “the person's face,” “theperson's eye,” or “the person's hand”) of the second image, which isconsidered to include personal information, in security information, andmay transmit the same to the electronic device 101. According to oneembodiment, the security information may include location information(e.g. pixel coordinates) of at least one image segment considered toinclude personal information and/or identification information (e.g.“the person's face,” “the person's eye,” or “the person's hand”) of eachof image segments.

According to an embodiment, the external electronic device 300 maytransmit correction segment information (e.g. the correction segmentinformation 332) to the electronic device 101 together with the securityinformation of the second image.

In operation 550, according to an embodiment, the processor 120 of theelectronic device 101 may secure the first image, at least partiallybased on the security information.

According to an embodiment, the processor 120 may divide, at leastpartially based on the security information, the first image into afirst security level segment and a second security level segment.

According to an embodiment, the processor 120 may generate a securitykey as a part of the method for securing the first image and store thesecurity key in the memory, may acquire a third image in which thesecond security level segment of the first image has been encoded basedon the security key, and may transmit the third image to the externalelectronic device 300. For example, the processor 120 may acquire thethird image by mosaicking, masking, or scrambling the second securitylevel segment of the first image. According to an embodiment, theprocessor 120 may receive, from the external electronic device 300, afourth image in which the first security level segment of the thirdimage has been at least partially corrected, and may acquire a fifthimage in which the fourth image has been decoded using the security keystored in the memory 130.

According to an embodiment, as a part of the method for securing thefirst image, the processor 120 may extract the first security levelsegment from the first image and generate the third image including thefirst security level segment, and may extract the second security levelsegment from the first image and generate a security segment imageincluding the second security level segment. According to an embodiment,the processor 120 may transmit the third image to the externalelectronic device 300 and may store the security segment image in thememory 130.

According to an embodiment, the processor 120 may receive, from theexternal electronic device 300, the fourth image in which the thirdimage has been at least partially corrected, and acquire the fifth imageby using the fourth image and the security segment image.

FIG. 6 is a flowchart illustrating a method for securing a first imagein an electronic device according to an embodiment. The operations inFIG. 6 may be included as a part of operation 550 or the entiretythereof in FIG. 5.

Referring to FIG. 6, in operation 610, according to an embodiment, theprocessor 120 of the electronic device 101 may acquire a third image inwhich a second security level segment of the first image has beenencoded based on the security key.

According to an embodiment, security information included in a secondimage received from the external electronic device 300 (e.g. theelectronic device 104 or the server 108 in FIG. 1 or the firstelectronic device 470 in FIG. 4) may include security recipe informationused to divide the second image into a first security level segmentand/or a second security level segment requiring higher security thanthe first security level segment. The processor 120 may divide, at leastpartially based on the security information or the security recipeinformation, the first image into a first security level segment and asecond security level segment. According to an embodiment, the firstimage may be divided into multiple security segments (e.g. threesecurity level segments) according to the security information or thesecurity recipe information.

According to an embodiment, the processor 120 may provide, based on thesecurity information received from the external electronic device 300, auser with an image indicating at least one security segment, and maydetermine the first security level segment and the second security levelsegment on the basis of a user input selecting a part of the at leastone security segment or the entirety thereof. For example, the processormay display at least one security segment (e.g. “a person's face,” “aperson's eye,” or “a person's hand) in the provided image, and, if theuser selects a specific segment (e.g. “the person's eye”), may designatethe selected specific segment (e.g. “the person's eye”) as the secondsecurity level segment.

According to an embodiment, the processor 120 may receive a user inputdesignating another segment (e.g. a segment of the image showing adocument) other than the at least one security segment within theprovided image, and may determine the designated another segment as thesecond security level segment.

According to an embodiment, the processor 120 may generate a securitykey and then may store the security key in the memory. The processor 120may generate the encoded third image from the first image by using thesecurity key and a hash function.

In operation 620, according to an embodiment, the processor 120 of theelectronic device 101 may transmit the third image to the externalelectronic device 300 (e.g. the electronic device 104 or the server 108in FIG. 1, or the first electronic device 470 in FIG. 4) through thecommunication module (e.g. the communication module 190 in FIG. 1). Forexample, the third image may have the second security level segmentwhich has been mosaicked, masked, and scrambled.

In operation 630, according to an embodiment, the processor 120 of theelectronic device 101 may receive a fourth image, in which the firstsecurity level segment of the third image is corrected, from theexternal electronic device 300 through the communication module (e.g.the communication module 190 in FIG. 1).

According to an embodiment, the external electronic device 300 maycorrect at least a part of the first security level segment. Forexample, the external electronic device 300 may correct the firstsecurity level segment by using the existing generated correctionsegment information (e.g. the correction segment information 332), ormay correct the first security level segment by using extendedcorrection segment information. The external electronic device 300 hasno security key used for the third image, and thus may not determine thesecond security level segment. For example, the external electronicdevice 300 may not correct the second security level segment.

In operation 640, according to an embodiment, the processor 120 of theelectronic device 101 may decode the fourth image on the basis of thesecurity key stored in the memory 130 and then acquire the fifth image.

According to an embodiment, the processor 120 may acquire the fifthimage only by decoding the fourth image. For example, the processor 120may demosaic, unmask, or unscramble the second security level segment inthe fourth image. According to an embodiment, the processor 120 maydecode the fourth image and then correct the second security levelsegment using the correction segment information (e.g. the correctionsegment information 332), thereby acquiring the fifth image. The fifthimage may be stored in the memory 130.

FIG. 7 is a flowchart illustrating a method for securing a first imagein an electronic device according to an embodiment. The operations inFIG. 7 may be included as a part of operation 550 or the entiretythereof in FIG. 5.

Referring to FIG. 7, in operation 710, according to an embodiment, theprocessor 120 of the electronic device 101 may extract a first securitylevel segment from a first image and generate a third including thefirst security level segment.

According to an embodiment, security information included in a secondimage received from the external electronic device 300 (e.g. theelectronic device 104 or the server 108 in FIG. 1 or the firstelectronic device 470 in FIG. 4) may include security recipe informationused to divide the second image into a first security level segmentand/or a second security level segment requiring higher security thanthe first security level segment. The processor 120 may divide, at leastpartially based on the security information or the security recipeinformation, the first image into the first security level segment andthe second security level segment. According to an embodiment, thesecurity information may be used to divide the first image into multiplesecurity segments (e.g. three security level segments)

According to an embodiment, the processor 120 may provide, based on thesecurity information received from the external electronic device 300, auser with an image indicating at least one security segment, and mayseparate the first security level segment and the second security levelsegment from each other on the basis of a user input selecting a part ofthe at least one security segment or the entirety thereof. For example,the processor may display at least one security segment (e.g. “aperson's face,” “a person's eye,” or “a person's hand) in the providedimage, and, if the user selects a specific segment (e.g. “the person'seye”), may designate the selected specific segment (e.g. “the person'seye”) as the second security level segment.

According to an embodiment, the processor 120 may receive a user inputdesignating another segment (e.g. a portion of the image showing adocument) other than the at least one security segment within theprovided image, and may determine the designated another segment as thesecond security level segment.

According to an embodiment, the processor 120 may extract the firstsecurity segment, at least partially based on the security information.For example, the third image may include only the first security segmentwithin the first image.

In operation 720, according to an embodiment, the processor 120 of theelectronic device 101 may extract the second security level segment fromthe first image and generate a security segment image including thesecond security level segment.

According to an embodiment, the processor 120 may extract the secondsecurity level segment, at least partially based on the securityinformation. For example, the security segment image may include onlythe second security level segment within the first image. In thisexample, the security segment image may be removed from the third image.

In operation 730, according to an embodiment, the processor 120 of theelectronic device 101 may transmit the third image to the externalelectronic device 300 through the communication module (e.g. thecommunication module 190 in FIG. 1).

In operation 740, according to an embodiment, the processor 120 of theelectronic device 101 may receive a fourth image, in which at least apart of the third image is corrected, from the external electronicdevice 300 through the communication module (e.g. the communicationmodule 190 in FIG. 1).

According to an embodiment, the external electronic device 300 maycorrect at least a part of the third image including the first securitysegment. For example, the external electronic device 300 may correct thethird image by using the existing generated correction segmentinformation (e.g. the correction segment information 332), or maycorrect the third image by using extended correction segmentinformation.

In operation 750, according to an embodiment, the processor 120 of theelectronic device 101 may acquire a fifth image by using the fourthimage and the security segment image.

According to an embodiment, the processor 120 may acquire the fifthimage by synthesizing the fourth image and the security segment image.According to one embodiment, the processor 120 may correct the securitysegment image including the second security level segment by using thecorrection segment information (e.g. the correction segment information332), and may acquire the fifth image by synthesizing the fourth imageand the corrected security segment image. The fifth image may be storedin the memory 130.

According to an embodiment, a method for securing personal informationincluded in an image may include: acquiring a first image including anexternal object by using a camera that includes an image sensor(operation 510); acquiring a second image including less informationthan the first image by using at least a part of the first image(operation 520); transmitting the second image to an external electronicdevice by using a communication module (operation 530); receivingsecurity information corresponding to the second image from the externalelectronic device by using the communication module (operation 540); andsecuring the first image, at least partially based on the securityinformation (operation 550).

According to an embodiment, the security information corresponding tothe second image may relate to at least one biometric informationsegment of the external object.

According to an embodiment, the securing (operation 550) may includedividing, at least partially based on the security information, thefirst image into a first security level segment and a second levelsecurity segment.

According to an embodiment, the method for securing personal informationincluded in an image may include: generating a security key and storingthe security key in a memory; acquiring a third image in which thesecond security level segment of the first image has been encoded basedon the security key (operation 610); and transmitting the third image tothe external electronic device (operation 620).

According to an embodiment, the method for securing personal informationincluded in an image may include: receiving, from the externalelectronic device, a fourth image in which the first security levelsegment of the third image has been at least partially corrected(operation 630); and acquiring a fifth image in which the fourth imagehas been decoded using the security key stored in the memory (operation640).

According to an embodiment, the method for securing personal informationincluded in an image may include: extracting the first security levelsegment from the first image and generating the third image includingthe first security level segment (operation 710); and extracting thesecond security level segment from the first image and generating asecurity segment image based on the second security level segment(operation 720).

According to an embodiment, the method for securing personal informationincluded in an image may include: transmitting the third image to theexternal electronic device; and storing the security segment image inthe memory.

According to an embodiment, the method for securing personal informationincluded in an image may include: receiving, from the externalelectronic device, the fourth image in which the third image has been atleast partially corrected (operation 740); and acquiring the fifth imageby synthesizing the fourth image and the security segment image(operation 750).

FIG. 8 is a flowchart illustrating a method for securing personalinformation included in an image in a system according to an embodiment.

Referring to FIG. 8, in operation 811, according to an embodiment, theelectronic device 101 may acquire a first image including an externalobject by using an image sensor (e.g. the camera module 180 in FIG. 1,the image sensor 230 in FIG. 2, the image sensor 321 in FIG. 3, or theimage sensor 411 in FIG. 4).

In operation 813, according to an embodiment, the electronic device 101may acquire a second image including less information than the firstimage by using at least a part of the first image.

In operation 815, according to an embodiment, the electronic device 101may transmit the second image to the external electronic device 300(e.g. the electronic device 104 or the server 108 in FIG. 1 or the firstelectronic device 470 in FIG. 4) through a communication module.

In operation 817, according to an embodiment, the external electronicdevice 300 may extract, based on the second image, at least one imagesegment (e.g. a security segment) considered to include personalinformation. For example, the external electronic device 300 may applyan object recognition algorithm or a text recognition algorithm to thesecond image so as to identify an object (e.g. a person) or recognize aspecific segment of the object (e.g. “a person's face,” “a person'seye,” or “a person's hand”).

In operation 819, according to an embodiment, the external electronicdevice 300 may transmit security information included in the secondimage to the electronic device 101. According to one embodiment, thesecurity information may include location information (e.g. pixelcoordinates) of the at least one image segment considered to includepersonal information or identification information of each of the imagesegments (e.g. “the person's face,” “the person's eye,” or “the person'shand”).

In operation 821, according to an embodiment, the electronic device 101may acquire third image at least partially based on the first image andthe security information. According to an embodiment, the electronicdevice 101 may acquire the third image in which a second security levelsegment of the first image has been encoded based on a security key.According to an embodiment, the electronic device 101 may extract afirst security level segment from the first image and may generate thethird image including the first security level segment.

In operation 823, according to an embodiment, the electronic device 101may transmit the third image to the external electronic device 300through the communication module (e.g. the communication module 190 inFIG. 1).

In operation 825, according to an embodiment, the external electronicdevice 300 may correct at least a part of the third image including thefirst security level segment to acquire a fourth image.

In operation 827, according to an embodiment, the external electronicdevice 300 may transmit the fourth image to the electronic device 101.

In operation 829, according to an embodiment, the electronic device 101may acquire a fifth image by using the fourth image. According to anembodiment, the processor 120 may acquire the fifth image only bydecoding the fourth image. According to another embodiment, theprocessor 120 may acquire the fifth image by synthesizing the fourthimage and the security segment image. The fifth image may be stored inthe memory 130 of the electronic device 101.

According to an embodiment, an electronic device (e.g. the electronicdevice 104 or the server 108 in FIG. 1, the external electronic device300 in FIG. 3, or the first electronic device 470 in FIG. 4) mayinclude: a communication module; a memory (e.g. the storage 335 in FIG.3); and a processor (e.g. the ISP 333 in FIG. 3 or the processor 471 inFIG. 4) operationally connected to the communication module and thememory. The memory stores instructions that, when executed, cause theprocessor to: receive an image from an external electronic device (e.g.the electronic device 101 in FIG. 1 or the second electronic device 400in FIG. 4) through the communication module; recognize at least oneimage segment in the image; determine security information included inthe image and associated with the at least one image segment, at leastpartially based on a result of the recognition; and transmit thesecurity information to the external electronic device through thecommunication module.

FIG. 9A illustrates an example of a method for securing personalinformation according to an embodiment, FIG. 9B illustrates an exampleof a method for securing personal information according to anembodiment, and FIG. 9C illustrates an example of a method for securingpersonal information according to an embodiment.

FIG. 9A illustrates a second image 901 (e.g. the small raw image 326 inFIG. 3) including an external object 911. By using at least a part of afirst image including the external object 911, the processor 120 mayacquire the second image 901 that includes less information than thefirst image.

FIG. 9B illustrates an image 921 provided to indicate at least onesecurity segment. On the basis of security information received from theexternal electronic device 300, the processor 120 may provide a userwith the image 921 indicating the at least one security segment (e.g. “aperson's eye 931,” “a person's ear 933,” or “a person's hand 935”).According to an embodiment, the processor 120 may divide the first imageinto a first security level segment and a second security level segmentwithout providing the image 921 to the user. For example, the processormay designate, as the second security level segment, segmentscorresponding to the “person's eye 931,” the “person's ear 933,” or the“person's hand 935.”

FIG. 9C illustrates a third image 941 in which the second security levelsegment of the first image has been encoded based on a security key.According to one embodiment, the processor 120 may determine a userinput selecting a part or the entirety of the at least one securitysegment provided through the provided image, and may separate the firstsecurity level segment and the second security level segment from eachother. For example, if the user select a specific segment (e.g. theperson's eye 931) from among the at least one security segment (e.g. the“person's eye 931,” the “person's ear 933,” or the “person's hand 935”)provided in the image, the processor 120 may determine the segmentcorresponding to the “person's eye 931” as the second security levelsegment. Further, according to an embodiment, the processor 120 mayacquire the third image 941 after dividing the first image into thefirst security level segment and the second security level segment.According to one embodiment, the third image 941 may have a secondsecurity level segment which has been mosaicked, masked (951), orscrambled.

The electronic device according to various embodiments may be one ofvarious types of electronic devices. The electronic devices may include,for example, a portable communication device (e.g., a smartphone), acomputer device, a portable multimedia device, a portable medicaldevice, a camera, a wearable device, or a home appliance. According toan embodiment of the disclosure, the electronic devices are not limitedto those described above.

It should be appreciated that various embodiments of the presentdisclosure and the terms used therein are not intended to limit thetechnological features set forth herein to particular embodiments andinclude various changes, equivalents, or replacements for acorresponding embodiment. With regard to the description of thedrawings, similar reference numerals may be used to refer to similar orrelated elements. It is to be understood that a singular form of a nouncorresponding to an item may include one or more of the things, unlessthe relevant context clearly indicates otherwise. As used herein, eachof such phrases as “A or B,” “at least one of A and B,” “at least one ofA or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least oneof A, B, or C,” may include any one of, or all possible combinations ofthe items enumerated together in a corresponding one of the phrases. Asused herein, such terms as “1st” and “2nd,” or “first” and “second” maybe used to simply distinguish a corresponding component from another,and does not limit the components in other aspect (e.g., importance ororder). It is to be understood that if an element (e.g., a firstelement) is referred to, with or without the term “operatively” or“communicatively”, as “coupled with,” “coupled to,” “connected with,” or“connected to” another element (e.g., a second element), it means thatthe element may be coupled with the other element directly (e.g.,wiredly), wirelessly, or via a third element.

As used herein, the term “module” may include a unit implemented inhardware, software, or firmware, and may interchangeably be used withother terms, for example, “logic,” “logic block,” “part,” or“circuitry”. A module may be a single integral component, or a minimumunit or part thereof, adapted to perform one or more functions. Forexample, according to an embodiment, the module may be implemented in aform of an application-specific integrated circuit (ASIC).

Various embodiments as set forth herein may be implemented as software(e.g., the program 140) including one or more instructions that arestored in a storage medium (e.g., internal memory 136 or external memory138) that is readable by a machine (e.g., the electronic device 101).For example, a processor (e.g., the processor 120) of the machine (e.g.,the electronic device 101) may invoke at least one of the one or moreinstructions stored in the storage medium, and execute it, with orwithout using one or more other components under the control of theprocessor. This allows the machine to be operated to perform at leastone function according to the at least one instruction invoked. The oneor more instructions may include a code generated by a compiler or acode executable by an interpreter. The machine-readable storage mediummay be provided in the form of a non-transitory storage medium. Wherein,the term “non-transitory” simply means that the storage medium is atangible device, and does not include a signal (e.g., an electromagneticwave), but this term does not differentiate between where data issemi-permanently stored in the storage medium and where the data istemporarily stored in the storage medium.

According to an embodiment, a method according to various embodiments ofthe disclosure may be included and provided in a computer programproduct. The computer program product may be traded as a product betweena seller and a buyer. The computer program product may be distributed inthe form of a machine-readable storage medium (e.g., compact disc readonly memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded)online via an application store (e.g., PlayStore™), or between two userdevices (e.g., smart phones) directly. If distributed online, at leastpart of the computer program product may be temporarily generated or atleast temporarily stored in the machine-readable storage medium, such asmemory of the manufacturer's server, a server of the application store,or a relay server.

According to various embodiments, each component (e.g., a module or aprogram) of the above-described components may include a single entityor multiple entities. According to various embodiments, one or more ofthe above-described components may be omitted, or one or more othercomponents may be added. Alternatively or additionally, a plurality ofcomponents (e.g., modules or programs) may be integrated into a singlecomponent. In such a case, according to various embodiments, theintegrated component may still perform one or more functions of each ofthe plurality of components in the same or similar manner as they areperformed by a corresponding one of the plurality of components beforethe integration. According to various embodiments, operations performedby the module, the program, or another component may be carried outsequentially, in parallel, repeatedly, or heuristically, or one or moreof the operations may be executed in a different order or omitted, orone or more other operations may be added.

Certain of the above-described embodiments of the present disclosure canbe implemented in hardware, firmware or via the execution of software orcomputer code that can be stored in a recording medium such as a CD ROM,a Digital Versatile Disc (DVD), a magnetic tape, a RAM, a floppy disk, ahard disk, or a magneto-optical disk or computer code downloaded over anetwork originally stored on a remote recording medium or anon-transitory machine readable medium and to be stored on a localrecording medium, so that the methods described herein can be renderedvia such software that is stored on the recording medium using a generalpurpose computer, or a special processor or in programmable or dedicatedhardware, such as an ASIC or FPGA. As would be understood in the art,the computer, the processor, microprocessor controller or theprogrammable hardware include memory components, e.g., RAM, ROM, Flash,etc. that may store or receive software or computer code that whenaccessed and executed by the computer, processor or hardware implementthe processing methods described herein.

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the present disclosure as defined by the appendedclaims and their equivalents.

What is claimed is:
 1. An electronic device comprising: a cameraincluding an image sensor; a communication module; a memory; and aprocessor operationally connected to the camera, the communicationmodule, and the memory, wherein the memory stores instructions that,when executed, cause the processor to: acquire a first image includingan external object by using the camera; acquire a second image includingless information than the first image by using at least a part of thefirst image; transmit the second image to an external electronic deviceby using the communication module; receive security informationcorresponding to the second image from the external electronic device byusing the communication module; and secure the first image at leastpartially based on the security information.
 2. The electronic device ofclaim 1, wherein the security information corresponding to the secondimage relates to at least one biometric information segment of theexternal object.
 3. The electronic device of claim 1, wherein theinstructions further cause the processor to divide, at least partiallybased on the security information, the first image into a first securitylevel segment and a second security level segment.
 4. The electronicdevice of claim 3, wherein the instructions further cause the processorto: generate a security key and store the security key in the memory;acquire a third image in which the second security level segment of thefirst image is encoded based on the security key; and transmit the thirdimage to the external electronic device.
 5. The electronic device ofclaim 4, wherein the instructions further cause the processor to:receive, from the external electronic device, a fourth image in whichthe first security level segment of the third image is at leastpartially corrected; and acquire a fifth image in which the fourth imageis decoded using the security key stored in the memory.
 6. Theelectronic device of claim 4, wherein the instructions further cause theprocessor to acquire the third image by mosaicking, masking, orscrambling the second security level segment of the first image.
 7. Theelectronic device of claim 4, wherein the instructions further cause theprocessor to: extract the first security level segment from the firstimage and generate the third image including the first security levelsegment; and extract the second security level segment from the firstimage and generate a security segment image based on the second securitylevel segment.
 8. The electronic device of claim 7, wherein theinstructions further cause the processor to: transmit the third image tothe external electronic device; and store the security segment image inthe memory.
 9. The electronic device of claim 8, wherein theinstructions further cause the processor to: receive, from the externalelectronic device, a fourth image in which the third image is at leastpartially corrected; and acquire a fifth image by synthesizing thefourth image and the security segment image.
 10. The electronic deviceof claim 3, wherein the instructions further cause the processor to:provide, based on the received security information, a user with animage indicating the first security level segment and the secondsecurity level segment; and separate the first security level segmentand the second security level segment from each other, based on a userinput selecting a part or an entirety of the first security levelsegment or the second security level segment.
 11. The electronic deviceof claim 10, wherein the instructions further cause the processor to:receive another user input designating another segment other than thefirst security level segment and the second security level segment inthe provided image; and determine the designated another segment as thesecond security level segment.
 12. A method for securing personalinformation included in an image, the method comprising: acquiring afirst image including an external object by using a camera that includesan image sensor; acquiring a second image including less informationthan the first image by using at least a part of the first image;transmitting the second image to an external electronic device by usinga communication module; receiving security information corresponding tothe second image from the external electronic device by using thecommunication module; and securing the first image, at least partiallybased on the security information.
 13. The method of claim 12, whereinthe security information corresponding to the second image relates to atleast one biometric information segment of the external object.
 14. Themethod of claim 12, wherein the securing of the first image comprisesdividing, at least partially based on the security information, thefirst image into a first security level segment and a second levelsecurity segment.
 15. The method of claim 14, further comprising:generating a security key and storing the security key in a memory;acquiring a third image in which the second security level segment ofthe first image is encoded based on the security key; and transmittingthe third image to the external electronic device.
 16. The method ofclaim 15, further comprising: receiving, from the external electronicdevice, a fourth image in which the first security level segment of thethird image is at least partially corrected; and acquiring a fifth imagein which the fourth image is decoded using the security key stored inthe memory.
 17. The method of claim 15, further comprising: extractingthe first security level segment from the first image and generating thethird image including the first security level segment; and extractingthe second security level segment from the first image and generating asecurity segment image based on the second security level segment. 18.The method of claim 17, further comprising: transmitting the third imageto the external electronic device; and storing the security segmentimage in the memory.
 19. The method of claim 18, further comprising:receiving, from the external electronic device, a fourth image in whichthe third image is at least partially corrected; and acquiring a fifthimage by synthesizing the fourth image and the security segment image.20. An electronic device comprising: a communication module; and amemory; and a processor operationally connected to the communicationmodule and the memory, wherein the memory stores instructions that, whenexecuted, cause the processor to: receive an image from an externalelectronic device by using the communication module; recognize at leastone image segment in the image; determine security information includedin the image and associated with the at least one image segment, atleast partially based on a result of the recognition; and transmit thesecurity information to the external electronic device by using thecommunication module.